Papers with statistical analysis
Building a Dialogue Corpus Annotated with Expressed and Experienced Emotions (2022.acl-srw)
Copied to clipboard
| Challenge: | a human would recognize the emotion of an interlocutor and respond with an appropriate emotion, such as empathy and comfort. |
| Approach: | They propose to build a dialogue corpus annotated with two kinds of emotions . they collect tweets and annotate them with the emotion they put into the utterance . |
| Outcome: | The proposed method shows that it is difficult to recognize experienced emotions and multitask learning is effective. |
Group, Extract and Aggregate: Summarizing a Large Amount of Finance News for Forex Movement Prediction (D19-51)
Copied to clipboard
| Challenge: | Existing studies on forex prediction ignore related text completely and focus on forex trade data only, which loses important semantic information. |
| Approach: | They propose a BERT-based Hierarchical Aggregation Model to summarize forex news . they group news from different aspects and extract the most crucial news in each group . |
| Outcome: | The proposed model outperforms baseline methods and grouping methods and summarizes the influence patterns for forex trading. |
NESTLE: a No-Code Tool for Statistical Analysis of Legal Corpus (2024.eacl-demo)
Copied to clipboard
| Challenge: | a comprehensive statistical analysis of legal corpus requires specialized tools or programming skills. |
| Approach: | They propose a no-code tool for large-scale statistical analysis of legal corpus . NESTLE can extract any type of information that has not been predefined in the IE system . |
| Outcome: | The proposed tool can perform comparable to LexGLUE on 15 Korean precedent IE tasks and 3 legal text classification tasks. |
Simplifying Outcomes of Language Model Component Analyses with ELIA (2026.eacl-demo)
Copied to clipboard
| Challenge: | ELIA is an interactive web application that simplifies the outputs of various language model component analyses for a broader audience. |
| Approach: | They propose to use a vision-language model to automatically generate natural language explanations for the complex visualizations produced by these methods. |
| Outcome: | The proposed system integrates three key techniques and generates natural language explanations for complex visualizations. |
Emo Pillars: Knowledge Distillation to Support Fine-Grained Context-Aware and Context-Less Emotion Classification (2025.findings-acl)
Copied to clipboard
| Challenge: | a recent study shows that sentiment analysis datasets lack context in which an opinion was expressed and are limited by a few emotion categories. |
| Approach: | They propose to ground an LLM-based model into a corpus of narratives to generate stories-character-centered utterances with unique contexts over 28 emotion classes. |
| Outcome: | The proposed model generates non-repetitive story-character-centered utterances with unique contexts over 28 emotion classes. |
Detecting Heavy Rain Disaster from Social and Physical Sensor (C18-2)
Copied to clipboard
| Challenge: | Our system detects heavy rain disaster using social and physical sensors. |
| Approach: | They propose a system that detects heavy rain disaster by analyzing tweets and physical sensors. |
| Outcome: | The proposed system detects heavy rain disaster using social and physical sensors in Japan. |
An Environment for Relational Annotation of Political Debates (P19-3)
Copied to clipboard
| Challenge: | Scalable text analysis techniques can open corpora to new questions in computational social sciences and digital humanities. |
| Approach: | They describe a tool that allows annotating newspaper text with rich information about claims (demands) raised by politicians and other actors. |
| Outcome: | The MARDY tool realizes the complete workflow necessary for annotating a large newspaper text collection with rich information about claims (demands) raised by politicians and other actors. |
TMID: A Comprehensive Real-world Dataset for Trademark Infringement Detection in E-Commerce (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Annually, e-commerce platforms incur substantial financial losses due to trademark infringements. |
| Approach: | They propose a dataset to detect trademark infringement in merchant registrations . they use legal rules and contextual information from Alipay to gather contextual information with annotations from legal experts. |
| Outcome: | The proposed dataset is sourced from Alipay, one of the world’s largest e-commerce and digital payment platforms. |
Quantifying the Influence of Irrelevant Contexts on Political Opinions Produced by LLMs (2025.acl-srw)
Copied to clipboard
| Challenge: | Recent studies have examined the generation of large language models (LLMs) on subjective topics such as political opinions and attitudinal questionnaires. |
| Approach: | They use a Political Compass Test questionnaire to quantify how irrelevant information can systematically bias model opinions in specific directions. |
| Outcome: | The results show that even seemingly unrelated contexts alter model responses in predictable ways. |
VeriMinder: Mitigating Analytical Vulnerabilities in NL2SQL (2025.acl-demo)
Copied to clipboard
| Challenge: | Application systems using natural language interfaces to databases (NLIDBs) have democratized data analysis, but they are not without significant risks. |
| Approach: | They propose an interactive system that detects and mitigates cognitive biases in analytical questions by using contextual semantic mapping frameworks. |
| Outcome: | The proposed system detects and mitigates cognitive biases in analytical questions and generates high-quality, task-specific prompts. |
BanglaBook: A Large-scale Bangla Dataset for Sentiment Analysis from Book Reviews (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing literature on Bangla Sentiment Analysis (SA) has limited data and cross-domain adaptability. |
| Approach: | They present a large-scale dataset of Bangla book reviews with 158,065 samples . they employ a range of machine learning models to establish baselines including SVM, LSTM, and Bangla-BERT. |
| Outcome: | The proposed model improves performance over models that rely on manual features. |
A Character-Centric Creative Story Generation via Imagination (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing narrative generation models lack diversity and character depth, but they are inadequate for human creativity. |
| Approach: | They propose a novel story generation framework called CCI that leverages images to create stories that are diverse and creative in their themes and richer in content. |
| Outcome: | The proposed framework significantly improves various aspects of the stories’ creativity. |
EM-PERSONA: EMotion-assisted Deep Neural Framework for PERSONAlity Subtyping from Suicide Notes (2022.coling-1)
Copied to clipboard
| Challenge: | Suicide continues to be one of the significant causes of death worldwide . EMotion-assisted personality subtyping is a novel approach to identify personality traits from suicide notes . |
| Approach: | They propose to use a PERSONAlity Detection Framework to identify personality traits from suicide notes and annotate them using a benchmark dataset. |
| Outcome: | The proposed method outperforms baselines on comprehensive evaluation using multiple state-of-the-art systems. |
PaCoST: Paired Confidence Significance Testing for Benchmark Contamination Detection in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models are trained on vast amounts of data, which may unintentionally or intentionally include data from commonly used benchmarks. |
| Approach: | They propose a set of requirements that practical contamination detection methods should follow to effectively detect benchmark contamination in large language models. |
| Outcome: | The proposed method detects whether the model is significantly more confident under the original benchmark. |
Controlling Reading Ease with Gaze-Guided Text Generation (2026.eacl-long)
Copied to clipboard
| Challenge: | Using a gaze-based model, we generate texts with controllable reading ease. |
| Approach: | They propose a method that predicts gaze patterns to steer language model outputs towards eliciting certain reading behaviors by predicting eye-tracking measures. |
| Outcome: | The proposed method generates texts with controllable reading ease using eye-tracking with native and non-native speakers of English. |
Correct after Answer: Enhancing Multi-Span Question Answering with Post-Processing Method (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work focuses on designing specific methods or applying heuristic strategies to encourage models to predict more correct predictions. |
| Approach: | They propose a framework that uses a post-processing strategy to handle incorrect predictions. |
| Outcome: | The proposed framework significantly improves the Exact Match scores on multiple MSQA datasets. |
Enhancing Chat Language Models by Scaling High-quality Instructional Conversations (2023.emnlp-main)
Copied to clipboard
| Challenge: | a recent study validates the effectiveness of chat language models by fine-tuning instruction data. |
| Approach: | They propose to use a large-scale dataset of instructional conversations to fine-tune a conversational model on instruction data. |
| Outcome: | The proposed model outperforms open-source models in key metrics including scale, average length, diversity, coherence, etc. |
Neural Deepfake Detection with Factual Structure of Text (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to deepfake detection typically represent documents with coarse-grained representations, but they struggle to capture factual structures of documents. |
| Approach: | They propose a graph-based model that captures factual structures of documents for deepfake detection. |
| Outcome: | The proposed model improves strong base models built with RoBERTa on two public deepfake datasets. |
QFrBLiMP: a Quebec-French Benchmark of Linguistic Minimal Pairs (2026.findings-eacl)
Copied to clipboard
| Challenge: | Specifically, these minimal pairs are created by manually modifying sentences extracted from an official online resource maintained by a Québec government institution. |
| Approach: | They propose to use the Quebec-French Benchmark of Linguistic Minimal Pairs to evaluate LLMs’ linguistic knowledge of prominent grammatical phenomena in Quebec-french. |
| Outcome: | The proposed corpus evaluates LLMs’ linguistic knowledge of prominent grammatical phenomena in Quebec-French. |
Emosical: An Emotion-Annotated Musical Theatre Dataset (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Emosical provides rich emotion annotations for musical films by inferring the background story of the characters. |
| Approach: | They propose to use a multimodal dataset of musical films to generate annotated emotion tags for each sample by inferring the background story of the characters. |
| Outcome: | The proposed dataset provides rich emotion annotations for musical films by inferring the background story of the characters. |
MedNgage: A Dataset for Understanding Engagement in Patient-Nurse Conversations (2023.findings-acl)
Copied to clipboard
| Challenge: | Literature suggests that actively engaged patients are more likely to obtain the full benefits of an intervention and exhibit better outcomes. |
| Approach: | They propose to annotate a dataset of patient-nurse conversations about cancer symptom management using a new framework for patient engagement. |
| Outcome: | The proposed model predicts patient-nurse conversations from socio-affective and cognitive dimensions. |
EMONA: Event-level Moral Opinions in News Articles (2024.naacl-long)
Copied to clipboard
Yuanyuan Lei, Md Messal Monem Miah, Ayesha Qamar, Sai Ramana Reddy, Jonathan Tong, Haotian Xu, Ruihong Huang
| Challenge: | Recent work on news articles has focused on social media short texts, but little has explored moral sentiment within news articles. |
| Approach: | They propose to extract event-level moral opinions from news articles using a new dataset . they use annotated event-based moral opinions to analyze news articles . |
| Outcome: | The proposed dataset consists of 400 news articles containing over 10k sentences and 45k events, among which 9,613 events received moral foundation labels. |
Investigating the Influence of Bilingual MWU on Trainee Translation Quality (L18-1)
Copied to clipboard
| Challenge: | a method for automatic extraction of bilingual multiword units (BMWUs) from a parallel corpus has been shown to be useful for estimating human translation quality. |
| Approach: | They applied a method for automatic extraction of bilingual multiword units from a parallel corpus in order to investigate their contribution to translation quality in terms of adequacy and fluency. |
| Outcome: | The method is based on generalized additive modelling and it shows that normalized BMWU ratios can be useful for estimating human translation quality. |
RaFoLa: A Rationale-Annotated Corpus for Detecting Indicators of Forced Labour (2022.lrec-1)
Copied to clipboard
| Challenge: | Forced labour is the most common type of modern slavery, affecting at least 24.9 million people worldwide. |
| Approach: | They propose to annotate an English corpus for multi-class and multi-label forced labour detection using specialised data from specialised sources. |
| Outcome: | The proposed corpus consists of 989 news articles annotated according to risk indicators defined by the International Labour Organization (ILO). |
Logic-Consistency Text Generation from Semantic Parses (2021.findings-acl)
Copied to clipboard
| Challenge: | Text generation from semantic parses is challenging due to the complexity of the inner logic and the lack of automatic evaluation metrics for logic consistency. |
| Approach: | They propose a framework for logic consistent text generation from semantic parses that employs iterative training procedures and quality control. |
| Outcome: | The proposed framework enhances logic consistency and human evaluation on two benchmark datasets. |
The ComMA Dataset V0.2: Annotating Aggression and Bias in Multilingual Social Media Discourse (2022.lrec-1)
Copied to clipboard
Ritesh Kumar, Shyam Ratan, Siddharth Singh, Enakshi Nandi, Laishram Niranjana Devi, Akash Bhagat, Yogesh Dawer, Bornini Lahiri, Akanksha Bansal, Atul Kr. Ojha
| Challenge: | 59,152 comments are annotated with a hierarchical, fine-grained taget marking aggression and bias of various kinds on social media platforms. |
| Approach: | They propose to annotate a multilingual dataset with a hierarchical, fine-grained tagset marking different types of aggression and the "context" in which they occur. |
| Outcome: | The proposed dataset contains 59,152 comments in four languages, mostly code-mixed with English. |
QUAK: A Synthetic Quality Estimation Dataset for Korean-English Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | despite its high utility, there are limitations concerning manual QE data creation. |
| Approach: | They propose to generate a Korean-English QE dataset that is fully automatic . they find that the algorithm is more accurate and faster than manual QE . |
| Outcome: | The proposed datasets show that they scale up to 1.58M and 6.58M, respectively, and show that the results are significantly better when compared to the previous datasets. |
Cross Domain Classification of Education Talk Turns (2025.coling-main)
Copied to clipboard
| Challenge: | Prior research has focused on the annotation of conversational talk-turns within the classroom, offering a statistical analysis of the various types of discourse prevalent in these environments. |
| Approach: | They examine the generalizability and transferability of text classifiers trained to predict classroom discourse across educational domains by accompanying each talk turn with dialog-level context. |
| Outcome: | The proposed models exhibit high generalizability when training and test datasets originate from the same or similar domains. |
MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing datasets for dialogue summarization are limited to their small sizes and are built from a narrow domain. |
| Approach: | They propose a large-scale media interview dataset consisting of 463.6K transcripts with abstractive summaries. |
| Outcome: | The proposed dataset is larger and contains multi-party conversations from multiple domains. |
Efficient Pairwise Annotation of Argument Quality (2020.acl-main)
Copied to clipboard
| Challenge: | Especially crowdsourcing suffers from assessors having different reference frames to base their judgments on and task instructions being nondescript and therefore unhelpful in ensuring consistency. |
| Approach: | They propose an efficient annotation framework for argument quality that uses a stochastic transitivity model and an effective sampling strategy to infer high-quality labels. |
| Outcome: | The proposed model significantly outperforms existing annotation procedures and offers statistical insights into argument quality. |
Enhancing Emotion-Cause Pair Extraction in Conversations via Center Event Detection and Reasoning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Emotion-Cause Pair Extraction in Conversations (ECPEC) aims to identify emotion utterances and their corresponding cause utterrances in unannotated conversations. |
| Approach: | They propose a new method to identify emotion utterances and their corresponding cause utterrances in unannotated conversations by using a center event-aware graph. |
| Outcome: | The proposed model outperforms existing methods and achieves state-of-the-art performance across three benchmark datasets. |
FinCorpus-DE10k: A Corpus for the German Financial Domain (2024.lrec-main)
Copied to clipboard
| Challenge: | a predominantly German corpus of financial documents is available for the first time . financial text is characterized by a unique vocabulary with implications including sentiment analysis . |
| Approach: | They propose a predominantly German financial corpus comprising 12.5k PDF documents . they hope it will fill this gap and foster further research in the financial domain . |
| Outcome: | The proposed corpus is the first non-email German financial corpus available . it aims to provide insights into financial discourse in the German language and multilingually. |
TST: A Schema-Based Top-Down and Dynamic-Aware Agent of Text-to-Table Tasks (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to extract text content based on static table structures neglect to deal with precise inner-document evidence extraction and dynamic information such as multiple entities and events. |
| Approach: | They propose a dynamic content extraction agent framework that uses type recognition to extract context evidences with the conduction of domain schema sequentially. |
| Outcome: | The proposed framework exhibits state-of-the-art (SOTA) performance on a large dataset. |
Linguistic and Embedding-Based Profiling of Texts Generated by Humans and Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have focused on using LLMs to classify text as either human-written or machine-generated . |
| Approach: | They characterize human-written and machine-generated texts using a set of linguistic features across different linguistic levels such as morphology, syntax, and semantics. |
| Outcome: | The proposed model reveals that human-written texts exhibit simpler syntactic structures and more diverse semantic content. |
Universal Dependencies: Extensions for Modern and Historical German (2024.lrec-main)
Copied to clipboard
| Challenge: | a new UD treebank is being developed for Middle High German annotations . the annotation scheme is inconsistent with other treebanks for this period . |
| Approach: | They propose to extend the UD scheme for modern and historical German by a range of tokens . they propose to use a treebank that is the first UD treebank for Middle High German . |
| Outcome: | The proposed extensions relate in part to differences between arguments and modifiers . the proposed treebank is the first UD treebank for Middle High German . |
PerspectiveMod: A Perspectivist Resource for Deliberative Moderation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Human moderators in online discussions face a heterogeneous range of tasks that go beyond content moderation, or policing. |
| Approach: | They propose a dataset of online comments annotated for the question "Does this comment require moderation?" they aim to improve discussion quality by analyzing annotator perspectives and annotating their views. |
| Outcome: | The proposed model is unique in its intentional variation across the level of moderation experience embedded in the source data, the annotator profiles and the individuality of the annnotator. |
Liaozhai through the Looking-Glass: On Paratextual Explicitation of Culture-Bound Terms in Machine Translation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to explicitating culturally-embedded meaning have focused on in-text solutions, overlooking paratextual apparatus in the footnotes and endnotes employed by professional translators. |
| Approach: | They formalize Genette's (1987) theory of paratexts and evaluate expert-aligned paratext models . they find that LLM-generated paratext improves audience comprehension . |
| Outcome: | The proposed model improves the comprehension of the Chinese short story Liaozhai by using human evaluations. |